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ABSTRACT 


Advanced  Navy  sensors  will  operate  simultaneously  in  multiple  spectral 
bands  to  achieve  target  detection,  discrimination,  and  identification.  Color 
image  processing  is  necessary  to  guide  the  development  of  integrated 
imaging  sensors  using  multi- spectral  focal  plane  arrays  and  to  demonstrate 
the  capabilities  of  these  emerging  sensors.  This  paper  discusses  human 
visualization  techniques  which  best  transfer  multi- spectral  information  to 
the  observer  for  critical  search,  surveillance,  and  targeting  decisions.  Fused 
two-  and  three -band  color  video  is  presented  from  data  gathered  at  Fort  AP 
Hill  using  bore-sighted  cameras  (LMIRIS  uncooled  320*240  long  wave 
infrared.  Amber  Radiance  256*256  mid-wave  infrared,  and  Fairchild  image 
intensified  CCD  cameras). 


1.  Introduction 

Color  fusion  of  multiband  imagery,  for  human  visualization,  is  anticipated  to  provide  increased 
ability  to  detect,  recognize,  and  identify  targets.  The  fused  image  is  expected  to  provide  as  much  or  more 
information  than  any  single  band  image.  This  field  test  was  performed  to  study  color  fusion  of  uncooled 
long  wave  infrared  and  low  light  visible  imagery.  A  third  midwave  camera  was  added  to  study  3-color 
fusion.  The  field  test  was  conducted  at  Fort  A  P  Hill  in  April  1998.  The  collaboration  participants  were 
Lockheed  Martin  -  Orlando,  Lockheed  Martin  -  Fairchild,  Lockheed  Martin  Imaging  Systems,  PVP 
Advanced  EO  Systems,  the  Naval  Post  Graduate  School,  and  the  Naval  Research  Lab.  Recently  available 
cameras  and  a  novel  data  acquisition  system,  which  recorded  real-time  digital  and  analog  data,  are 
described.  Imagery  of  a  variety  of  targets  and  scenes  were  collected.  The  focus  of  the  field  test  did  not 
include  studying  optimal  band  combinations  or  individual  scenes  during  an  entire  day.  Representative 


imagery  is  presented  as  both  2-color  and  3-color  fused  images,  and  as  both  white-hot  and  black-hot. 
Fusion  as  a  correction  for  loss  of  information  due  to  blooming  and  fog  is  addressed.  Finally,  a  way  of 
measuring  the  improvement  of  the  fused  imagery  over  single-band  imagery,  ROC-curves,  is  presented. 


2.  Data  collection 

This  field  test  was  novel  in  that  data  was  recorded  digitally  from  three  bore-sighted  cameras. 
One-minute  sequences  of  real-time  imagery  could  be  recorded.  A  sub-goal  of  the  field  test  was  to  learn 
how  to  assemble  a  digital  data  collection  system  using  PC’s  and  frame-grabbers.  Fort  AP  Hill  provides  a 
variety  of  scenes  and  targets. 

2.1.  Cameras 

The  cameras  used  for  this  field  test  were  a  LMIRIS  uncooled  320*240  long  wave  infrared,  an  Amber 
Radiance  256*256  mid-wave  infrared,  and  a  Fairchild  image  intensified  CCD  camera.  The  cameras  were 
mounted  vertically  in  a  rigid  structure,  as  closely  as  possible,  to  minimize  parallax  in  the  horizontal 
direction.  The  cameras  were  bore-sighted  but  did  not  share  a  common  optic.  The  fields-of-view  of  the 
cameras  were:  long-wave  33.7  by  26.5,  mid-wave  22.4  degrees  by  22.4  degrees,  and  visible  40  degrees  by 
30  degrees.  In  the  final  fused  imagery,  the  camera  with  the  smallest  field-of-view,  the  Amber  camera, 
dictates  the  image  size.  No  common  optic  was  devised  because  off-line  NRL  in-house  algorithms  exist  to 
rubber-sheet  the  single  band  images  into  a  common  magnification  and  orientation. 

2.2.  Data  Acquisition 

The  data  acquisition  system  recorded  both  analog  and  digital  data.  Storing  a  one-minute  sequence  of  data 
allows  movement  to  be  including  in  target  detection  studies  and  rare  events  to  be  captured.  Also,  storing 
digital  data  allows  it  to  be  processed  computationally.  Analog  data  must  be  digitized  before  any 
computational  analysis  can  be  performed  with  it  and  is  only  8-bit,  so  any  information  over  8-bit  is  lost. 
Although,  storing  analog  data  is  easier,  more  reliable  and  does  not  require  huge  amounts  of  computer  disk 
space. 

To  record  analog  data,  RS-170  cables  from  all  three  cameras  were  recorded  to  Super  Hi  VHS-8  tapes.  To 
record  data  digitally,  RS-170  data  for  the  LWIR  and  visible  cameras,  and  RS-422  for  the  MWIR  camera, 
were  digitized  by  frame-grabbers  in  PC’s  and  then  stored  to  high-speed  hard  disks.  The  frame  grabbers 
were  Imaging  Technology  IC-PCI  cards  with  AM-FA  daughter  cards  to  digitize  the  RS-170  data  and  IC- 
PCI  mother  boards  and  AM-DIG  daughter  cards  to  digitize  the  RS-422  data.  Data  acquisition  software, 
EYE  Image  Calculator,  read  data  from  the  frame  grabbers  and  stored  the  data,  in  one-minute  sequences, 
in  real-time  via  a  SCSI  controller,  to  hard  drives.  Three  PC’s  were  used,  one  for  each  camera.  Each  PC 
ran  Eye  Image  Calculator  and  was  equipped  with  a  frame  grabber.  These  three  computers  were  networked 
to  a  fourth  computer,  a  server  with  30GB  of  disk  space,  to  which  the  data  could  be  transferred.  The  RS- 
422  recorded  digitally  from  the  Amber  Radiance  camera  was  corrupted  and  unusable.  The  analog 
recording  for  the  Amber  Radiance  was  digitized  after  the  field  test  for  all  midwave  data  that  was 
analyzed.  Each  one-minute  sequence,  of  all  three  cameras,  is  about  0.8  GBytes.  A  total  of  19  GB  of 
visible  and  longwave  digital  data  and  16  hours  of  VHS  tape  for  all  three  cameras  were  stored.  A  future 
improvement  to  the  data  acquisition  system  will  be  to  record  12-bit  RS-422  data  from  all  cameras. 


2.3.  Imagery  Scenarios 

Data  was  taken  in  five  different  scene  scenarios:  (1)  in  heavy  clutter,  woods,  (2)  in  low  clutter,  an  open 
field,  (3)  in  medium  clutter,  a  compound  of  buildings,  (4)  in  medium  clutter,  along  a  tree-line,  (5)  in 
medium  clutter,  near  a  set  of  buildings  and  trees.  The  data  was  collected  in  the  nighttime,  during  full 
moon,  and  at  dawn  and  dusk.  Military  targets  were  an  M2,  an  Ml  13  -  a  large  truck,  M60A3  -  a  tank,  an 
M35,  and  a  HMMWV.  People  and  cars  were  also  used  for  targets.  The  ranges  of  the  data  were  25  meters 
in  woods  scenes,  and  less  than  1  km  and  up  to  5  km  in  the  open  fields.  GPS  data  was  recorded  from  the 
vehicles.  Meteorology  data  also  recorded  during  the  field  test. 


3.  Phenomenology  and  image  creation 

The  cameras  were  chosen  to  explore  2-color  fusion  of  the  uncooled,  long-wave  infrared  and 
visible  bands.  The  third  mid-wave  camera  was  added  to  study  three-color  fusion,  since  human  color 
vision  maps  more  readily  to  a  three-color  system. 

The  two-band  data  does  not  readily  correspond  to  three-color  human  vision.  Instead,  since  human  vision 
employs  color-opponency,  the  two  bands  are  displayed  as  red  and  cyan,  opponent  colors  (Ref  1-2).  To 
display  the  images  on  a  RGB  24-bit  display,  the  longer  of  the  two  bands  is  written  to  the  red  color  display 
buffer.  The  shorter  of  the  two  bands  is  written  to  both  the  green  and  blue  buffers.  Green  and  blue  combine 
to  make  cyan.  In  this  system,  the  pixels  have  chrominant  values  that  go  from  red,  through  gray,  to  cyan. 

In  the  brightness  direction,  the  pixel  values  vary  from  black  to  white.  The  three-color  data  is  fused  as  the 
longest  band  to  the  red  buffer,  the  midwave  band  to  green,  and  the  visible  band  to  blue.  All  shades  of  red, 
green,  blue  and  any  combination  of  the  three,  plus  black,  all  shades  of  gray,  and  white  are  possible  final 
pixel  colors  in  the  3-color  display. 

An  improvement  that  can  be  made  to  this  simple  color  fusion  “algorithm”  is  to  find  the  principle  axis  of 
the  distribution  of  pixels  which  typically  lies  along  the  brightness-darkness  direction.  Then,  for  2-color 
fusion,  a  line  orthogonal  to  this  direction  is  the  color  axis.  For  3-color  fusion,  a  plane  orthogonal  to  this 
direction  describes  the  color  of  the  pixel.  Rotating  into  this  principle  component  space  allows  the  pixels  to 
be  normalized  so  as  to  maximize  differences  in  the  bands  and  to  represent  those  differences  as  pixel  color. 

3.1 .  Two-color  versus  Three-color  Fusion 

Imagery  was  fused  using  both  two-  and  three-color  fusion.  The  images  have  been  registered,  from  the  raw 
data  in  which  all  three  cameras  had  different  fields-of-view,  to  match  the  field-of-view  of  the  Amber 
camera  (Ref  1).  In  the  color  image  in  Figure  1,  visible  and  uncooled  longwave  infrared  images  are  fused. 
The  top  image  is  from  the  low  light  camera.  Blooming  from  the  Jeep  taillights  cause  the  front  of  the 
building  to  be  saturated.  The  lower  image  was  taken  with  an  uncooled  long  wave  camera  at  the  same 
time.  The  fused  image,  on  the  right,  shows  that  the  truck’s  lights  were  on,  from  the  blue  tint  of  the  pixels 
behind  the  Jeep,  but  the  windows  of  the  building  can  be  seen  using  the  infrared  information.  Fused 
imagery  allows  a  way  to  “look-through”  blooming. 

From  the  same  sequence,  stills  from  all  three  cameras  are  presented  in  Figure  2.  The  top  left  image  is  the 
visible  image,  the  top  middle  is  the  midwave  image,  and  the  top  right  is  the  longwave  image.  The  bottom 
images  are  3-color  fusion  of  the  top  images.  The  bottom  left  image  fuses  the  infrared  images  as  black-as- 
hot  and  the  bottom  right  image  fuses  the  infrared  images  as  white-as-hot.  The  mid-wave  infrared  image 
contributes  building  relief  to  the  fused  image  and  the  long-wave  infrared  image  contributes  the  dark  color 
of  the  vehicles’  windshields.  In  the  black-as-hot  fused  image,  the  Jeep’s  windshield  appears  red,  as  do  the 


vehicles  in  the  background  near  the  building  in  the  right  side  of  the  image.  The  Jeep,  an  extended  target, 
has  many  colors  itself,  reminding  us  that  when  a  target  with  a  certain  characteristic,  such  as  a  paint  color, 
is  searched  for,  many  pixels  on  that  target  will  be  made  of  additional  material  with  completely  different 
emission  and  absorption  characteristics.  A  thermal  scar  has  been  left  in  the  trail  of  the  Jeep. 

This  scene  is  also  fused  as  all  combinations  of  2-color  images  in  Figure  3.  The  left  column  is  fused 
longwave  infrared  and  visible  images,  the  middle  column  is  midwave  infrared  and  visible,  the  right 
column  is  longwave  infrared  and  midwave  infrared.  The  longer  of  the  two  bands  is  represented  as  red,  the 
shorter  as  cyan.  In  the  top  row,  the  infrared  images  are  black-as-hot,  and  in  the  bottom  row,  they  are 
white-as-hot.  Since  the  infrared  bands  are  highly  correlated,  the  2-color  fused  infrared  images  appear 
gray.  In  the  visible-infrared  images,  the  treeline  and  details  of  branches  and  lights  in  the  treeline  are 
apparent  from  the  visible  data.  The  thermal  scar  and  difference  in  temperature  of  the  building  and 
windows  are  apparent  from  the  infrared  imagery. 

To  quantify  the  improvement  of  target  detection  of  two-color  versus  three-color,  ROC-curves,  described 
in  Section  4,  are  calculated. 

3.2.  White-hot  versus  Black-hot  Fused  Imagery 

When  analyzing  the  data,  the  question  was  pursued,  “Is  the  target  easier  to  detect  if  white-hot  or  black-hot 
imagery  is  fused  with  the  visible  imagery”.  The  midwave  and  longwave  imagery,  especially  at  nighttime, 
is  almost  entirely  self-emitted.  The  two  infrared  bands  are  highly  correlated  and  uncorrelated  with  the 
visible  band.  In  Figure  4,  a  scene  of  a  person  in  woods,  the  single  images  and  the  three-color  fused  images 
show  the  difference  between  black-as-hot,  the  top  image,  and  white-as-hot,  the  bottom  image.  Since  the 
data  was  recorded  digitally,  it  is  possible  to  invert  the  image.  The  person  is  standing  in  front  of  a  tree 
trunk  in  the  bottom  center-right  of  the  image.  Even  though  the  person  is  very  detectable  in  the  midwave 
data,  the  ground  and  sky  are  both  medium-gray  in  the  midwave  image.  The  fused  images  show  good  color 
difference  between  the  sky,  trees,  and  ground.  These  images  can  be  presented  in  human  performance  tests 
(Ref.  3)  to  determine  if  viewers  prefer,  and  perform  target  detection  tasks  better,  if  the  imagery  is  shown 
as  white-as-hot  or  black-as-hot.  Figures  2,  3,  and  5  also  show  both  images  black-as-hot  and  white-as-hot 
fusion. 


3.3.  Fog  and  Color  Fusion 

An  image  sequence  of  a  military  convoy  was  recorded  when  there  was  substantial  fog  on  the  ground, 
early  in  the  morning.  The  fused  images  show  both  information  from  the  visible  band,  that  it  was  foggy, 
and  from  the  infrared  band,  that  targets  are  present.  Figure  5.  The  top  left  picture  was  taken  with  the  low 
light  visible  camera.  The  middle  image  is  from  the  midwave  infrared  camera.  The  top  right  image  is  from 
the  uncooled  long  wave  camera.  The  bottom  left  is  a  3-color  fused  image,  black-is-hot,  and  the  bottom 
right  is  a  3-color  fused,  white-is-hot,  image.  Each  single  band  image  lacks  information  that  the  fused 
images  have.  The  visible  image  does  not  show  the  targets  and  the  horizon.  The  midwave  and  longwave 
images  do  not  exhibit  the  fact  that  it  is  foggy.  The  longwave  image  does  not  represent  a  difference 
between  the  road  and  the  ground.  In  addition,  the  fused  image  shows  you  that  the  shades  of  color  in  the 
sky  are  different  from  the  shades  of  color  in  the  ground,  which  is  important  if  the  horizon  is  lost  to 
buildings  or  a  treeline. 


4.  ROC  curves 

Although  some  aspects  of  fusion  are  not  easily  measured,  one  quantitative  test  of  improvement  of 
single  band  imagery  over  fused  imagery  that  can  be  made  is  a  ROC,  risk-opportunity  curve.  A  ROC  curve 
is  a  plot  of  false  alarms  versus  missed  detections.  ROC-curves  are  calculated  by  setting  a  series  of 
thresholds  and  determining  what  fraction  of  pixels  are  above  and  below  the  thresholds.  False  alarms  are 
calculated  as  the  number  of  background  pixels  with  values  greater  than  the  threshold,  normalized  by  the 
total  number  of  background  pixels.  Missed  detections  are  calculated  as  the  difference  of  the  number  of 
target  pixels  minus  the  number  of  target  pixels  greater  than  the  threshold,  normalized  by  the  total  number 
of  target  pixels.  This  test  determines  the  uniqueness  of  pixel  color  or  intensity,  but  does  not  measure 
target  detection  due  to  target  shape.  Another  analysis,  matched  filtering,  uses  target  shape  as  a  detection 
variable. 

The  calculation  of  ROC-curves  representing  the  pixel  differences  of  one  vehicle  in  an  image  of  a 
convoy  of  military  vehicles  from  the  background.  Figure  6,  will  be  used  as  an  example.  The  top  left 
image  is  a  3-color  fused  image,  fused  with  the  simple  color  algorithm  (Ref  2-3),  of  the  longwave, 
midwave,  and  visible  bands.  These  3-color  and  a  2-color  images  are  prepared  to  create  ROC  curves.  The 
images  are  then  transformed  into  the  principle  component  space.  The  vehicles  that  are  not  chosen  as  the 
target  are  vetoed,  shown  in  black,  so  that  their  pixels  will  not  be  counted  as  false  alarms.  The  sky  is  also 
vetoed,  so  that  in  the  principle  component  space,  the  chromaticity  plane  can  be  filled  with  the  active 
pixels,  allowing  maximum  separation  of  the  vehicle  pixels  from  the  background  pixels.  The  angles  of 
rotation  calculated  to  rotate  into  the  principle  component  space,  without  the  vetoed  pixels,  can  be  used  to 
present  the  entire  image,  including  the  sky  and  other  vehicles. 

For  the  2-color  fused  image,  the  histogram  of  the  background  pixels  values  along  the  chromaticity  line, 
after  a  rotation  into  the  principle  component  space,  is  shown  in  the  top  plot  of  Figure  7.  In  the  bottom 
plot,  the  histogram  of  pixel  values  of  the  target  is  plotted.  Pixels  with  values  near  zero  have  little  color. 
Pixel  values  to  the  left  are  colored  cyan  and  pixel  values  to  the  right  are  colored  red.  The  target,  located  at 
about  -0.4  and  therefore  almost  fully  cyan,  is  overlapped  by  background  pixels. 

The  pixel  values  in  the  chromaticity  plane  after  a  rotation  into  the  principle  component  space,  are  shown 
in  Figure  8.  The  target  pixels  are  shown  as  red  points  between  the  angles  300  and  330  at  the  edge  of 
the  distribution.  There  are  some  background  pixels  with  the  same  radius,  or  color  saturation,  but  no 
background  pixels  have  the  same  angle,  or  color  hue,  and  saturation.  The  target  pixels  are  very  well 
separated  from  the  background  pixels. 

The  ROC  curves  are  plotted  in  Figure  9.  In  this  plot,  a  line  to  the  left  has  a  better  detection  performance 
than  a  line  to  the  right.  The  solid  lines  to  the  upper  right  of  the  image  are  the  single  band  curves.  For  the 
fusion  images,  ROC  curves  were  calculated  considering  only  saturation  and  again  considering  saturation 
and  hue.  For  the  2-color  images,  all  of  the  pixels  are  considered  for  the  saturation  curve,  but  only  the 
pixels  below  zero  are  included  in  the  hue  and  saturation  curve. 

The  midwave  camera  performs  better  than  the  uncooled  longwave  or  low-light  visible  cameras.  The 
single  band  imagery  was  usually  at  least  an  order  of  magnitude  worse  than  the  fused  imagery.  For  the  2- 
color  imagery,  a  target  with  very  bright  pixel  intensity  is  easily  detected,  but  hue  does  not  add  additional 
information  over  saturation,  since  there  are  few  background  pixels  with  large  red  (greater  than  zero) 
values.  For  the  3-color  fused  imagery,  hue  does  aid  in  separating  the  target  pixels  from  the  background 
pixels,  placing  them  in  a  unique  position  in  the  chromaticity  plane,  as  seen  in  the  polar  plot  in  Figure  8. 
The  ROC-curves  for  the  3-color  fused  imagery  shows  much  improved  target  detection  over  single  band 
imagery  when  considering  pixel  saturation  alone.  When  hue  and  saturation  are  considered,  the  ROC- 


curve  shows  another  order  of  magnitude  improvement  over  single  hand  and  2-eolor  fused  imagery.  This 
fact  shows  an  improvement  in  detection  of  3-color  over  2-color  fused  imagery. 


5.  Summary 

The  Fort  A  P  Hill  field  test  provided  good  imagery  to  study  2-color  fusion  of  longwave  uncooled 
and  visible  imagery  and  3-color  fusion  of  longwave,  midwave,  and  visible  imagery.  In  this  test,  real-time 
one-minute  sequences  were  recorded  digitally,  which  enables  the  data  to  be  processed  as  simple  color 
fusion  and  principle  eomponent  fusion  off-line.  The  questions  of  whether  white-as-hot  and  black-as-hot 
infrared  visible  imagery  and  imagery  is  better  for  detection  can  be  studied  with  this  data  set  using  human 
factors  testing.  Blooming  “look-through”  was  demonstrated.  A  hard-to-catch  scene  of  a  military  convey 
in  heavy  morning  fog  was  recorded.  ROC  curves  for  the  single  bands,  2-color  fused  images,  and  3-color 
fused  images  were  calculated.  The  single  band  imagery  was  usually  at  least  an  order  of  magnitude  worse 
than  the  fused  imagery.  For  the  2-eolor  imagery,  targets  with  very  bright  pixel  intensity,  i.e.  large 
saturation,  are  easily  detected,  but  hue  does  not  add  additional  information  over  saturation.  For  the  3-color 
fused  imagery,  hue  does  aid  in  separating  the  target  pixels  from  the  background  pixels,  placing  them  in  a 
unique  position  in  the  chromaticity  plane.  The  ROC-curves  for  the  3-color  fused  imagery  shows  much 
improved  target  detection  over  single  band  imagery  when  pixel  saturation  only  is  eonsidered.  When  hue 
and  saturation  are  considered,  the  ROC-curve  for  the  3-color  fused  imagery  shows  another  order  of 
magnitude  more  improvement  over  the  single  band  imagery  and  the  2-color  fused  imagery. 
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Figure  1.  The  top  image  is  from  a  low  light  camera.  Blooming  causes  the  windows  in  the  building  to  he 
saturated.  The  lower  image  was  taken  with  an  uncooled  long  wave  camera  at  the  same  time.  The  fused 
image,  on  the  right,  shows  that  the  truck’s  lights  came  on  hut  has  not  lost  information  due  to  blooming. 
Fused  imagery  allows  a  way  to  “look-through”  blooming. 


Figure  2.  In  this  scene  a  Jeep  Cherokee  is  driving  through  a  building  compound.  The  top  left  image  is  the 
visible  image,  the  top  middle  is  the  midwave  image,  and  the  top  right  is  the  longwave  image.  The  bottom 
images  are  3-color  fusion  of  the  top  images,  the  left  uses  the  infrared  as  black-as-hot  and  the  right  uses  the 
infrared  as  white-as-hot.  In  the  fused  images,  the  separation  of  treeline,  clouds  and  building  is  more 
apparent.  In  the  black-as-hot  fused  image,  the  vehicles  windshield  appears  red,  as  do  the  vehicles  in  the 
background  near  the  building  in  the  right  side  of  the  image. 


Figure  3.  All  2-color  fusion  combinations  of  the  three  single  bands  images  in  Figure  2  are  shown.  The 
fusion  algorithm  is  simple  color.  The  left  column  shows  fusion  of  the  uncooled  long  wave  infrared  image 
with  the  low  light  visible  image.  The  middle  column  fuses  midwave  infrared  with  visible  and  the  right 
column  fuses  longwave  infrared  with  midwave  infrared.  In  the  top  row,  the  images  are  fused  with  black- 
as-hot  infrared  data.  In  the  bottom  row,  the  images  are  fused  with  white-as-hot  infrared  data.  The  two 
infrared  bands  are  highly  correlated  so  that  the  final  images  appear  gray.  The  lights  in  the  trees  and  the 
tree  branches  from  the  visible  band  and  the  thermal  scar  from  the  infrard  bands  are  apparent  in  the 
infrared- visible  fused  images. 


Figure  4.  The  top  image  is  from  a  low  light  camera,  the  middle  from  a  midwave  infrared  camera,  and  the 
bottom  from  an  uncooled  long  wave  infrared  camera.  The  fused  images  are  hlack-as-hot  and  white-as-hot. 
Notice  the  sky  has  a  different  color  from  the  ground  and  the  trees. 


Figure  5.  The  top  left  picture  was  taken  with  a  low  light  visible  camera,  during  early  morning  fog.  The 
middle  image  is  from  a  midwave  infrared  camera.  The  top  right  image  is  from  a  uncooled  long  wave 
camera.  The  bottom  left  is  a  3-color  fused  image,  black-is-hot ,  and  the  right  is  a  3-color  fused,  white-as- 
hot  image.  Each  single  band  image  lacks  information  that  the  fused  images  have.  The  visible  image  lost 
the  targets  and  the  horizon.  The  midwave  and  longwave  images  lost  the  fact  that  it  is  foggy.  The 
longwave  image  lost  the  road  from  the  ground.  In  addition,  the  fused  image  shows  you  that  the  shades  of 
color  in  the  sky  are  different  from  the  shades  in  the  ground,  which  is  important  if  the  horizon  is  lost  to 
buildings  or  a  treeline. 


Figure  6.  The  top  left  image  is  a  3-color  fused  image,  fused  with  the  simple  color  algorithm,  of  the 
longwave,  midwave,  and  visible  hands.  The  image  shows  a  convey  of  military  vehicles,  located  in  the 
right-middle  of  the  image,  in  front  of  the  tree-line.  The  image  is  prepared  for  processing  to  calculate  a 
ROC-curve,  false  alarms  versus  missed  detections,  of  one  of  the  tanks  compared  to  the  background 
(Figure  8).  The  prepared  2-color  and  3-color  ,  fused  with  the  principle  component  fusion  algorithm,  are 
shown  in  the  top  right  and  bottom  left.  The  vehicles  that  are  not  chosen  as  the  target  are  vetoed,  shown  as 
black,  so  that  their  pixels  will  not  be  counted  as  false  alarms.  The  sky  is  also  vetoed,  so  that  in  the 
principle  component  space,  the  chromaticity  plane  can  be  filled  with  the  active  pixels,  allowing  maximum 
separation  of  the  vehicle  pixels  from  the  background  pixels.  The  angles  of  rotation  calculated  to  rotate 
into  the  principle  component  space,  without  the  vetoed  pixels,  can  be  used  to  present  the  entire  image, 
including  the  sky  and  other  vehicles. 
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Figure  7.  For  the  2-color  fused  image  in  Figure  5,  the  histogram  of  the  radius  of  the  background  pixels 
values,  along  the  chromaticity  line  after  a  rotation  into  the  principle  component  space,  are  plotted  in  the 
top  image.  Pixel  values  near  zero  have  little  color,  pixel  values  to  the  left  are  colored  purple  and  pixel 
values  to  the  right  are  colored  green.  In  the  bottom  plot,  the  pixel  values  of  the  target  pixels  are  plotted. 
The  target  pixels  are  overlapped  by  the  background  pixels. 


piitEite  and  taDkgrourd  pikPhP 


Figure  8.  For  the  3-color  fusion  image  in  Figure  5,  the  pixel  values  in  the  chromaticity  plane  after  a 
rotation  into  the  principle  component  space,  are  shown.  The  target  pixels  are  shown  as  points  between 
the  angles  300  and  330  at  the  edge  of  the  distribution.  There  are  some  background  pixels  with  the  same 
radius,  or  color  saturation,  but  no  background  pixels  have  the  same  angle,  or  color  hue,  and  saturation. 
The  target  pixels  are  very  well  separated  from  the  background  pixels. 
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Figure  9.  In  this  Roc  curve  for  Figure  5,  a  line  to  the  left  has  a  better  detection  performance  than  a  line  to 
the  right.  The  solid  lines  to  the  upper  right  of  the  image  are  the  single  hand  curves.  The  midwave  camera 
performs  better  than  the  uncooled  longwave  or  low-light  visible  cameras.  For  the  fusion  images,  roc  curve 
were  calculated  considering  only  saturation  and  considering  saturation  and  hue.  For  the  2c-fusion  image, 
the  saturation  only  and  hue  and  saturation  curves  are  identical.  For  the  3-color  fusion  image,  making  a  cut 
on  hue  greatly  improves  detection. 


